Methods and apparatus for designing the racking and wiring configurations for pieces of hardware

ABSTRACT

In one embodiment of the invention, a method of designing the racking configuration for boxes in racks and for determining which connections go between different racks, including: solving a rack select optimization sub-problem to determine racks to use; and solving a rack assign optimization sub-problem to determine which particular rack will hold a particular box. In another embodiment, an apparatus for designing the racking configuration for boxes in racks and for determining which connections go between different racks, including: a machine-readable representation for a racking configuration problem; and a solver that can read that machine-readable representation and that is configured to: solve a rack select optimization sub-problem to determine at least one rack to use; and solve a rack assign optimization sub-problem to determine which particular rack will hold at least one box.

TECHNICAL FIELD

[0001] Embodiments of the present invention relate generally to methodsand apparatus for designing the racking and wiring configurations forpieces of hardware such as, for example, networked devices.

BACKGROUND

[0002] The rack configuration problem is defined as follows. We aregiven a set of boxes, a set of racks (e.g., bookcase-like structures),and a set of places to put the racks. Some or all of the boxes may haveone or more connections or links to other boxes. The racks may beextant, or may merely be a set of available rack types that could beused, perhaps with limits on the number available. The placement ofboxes in racks is constrained by physical limits such as their size (arack holds only a certain amount or number of boxes), power (a rack canonly support a certain power drain from the boxes it holds), and soforth. The objective is to decide where to position the boxes into theracks and how to place the racks themselves, subject to theseconditions, in such a way as to minimize various objectives, such as thenumber or cost of the racks used, the total length of the links betweenracks, or the number of links that cross rack boundaries.

[0003] In the most general case, a rack can be any container of boxes, abox can be any entity that can be placed in a rack.

[0004] A particular example of interest occurs when the boxes arecomputer or network components, such as network devices, computers,storage devices, hubs, switches, routers, displays, keyboards, storagearea network (SAN) devices, fans, air ducts, telecommunication devices,telephone equipment such as telephones, computer system components,Internet data center devices, local area network (LAN) devices, diskarray components, tape library devices, UNIX system components, WINDOWSsystem components, I/O subsystems, I/O and storage controllers, powersupplies, cooling units, and other electronic devices. In what follows,the terms “box”, “device”, and “component” will be used to encompass allof these kinds of boxes, as well as any other item that could be placedin a rack subject to constraints such as the rack's capacity.

[0005] In the computer network case, the racks are often standardcomputer-system mounting racks, which typically hold boxes that aredesigned with one or two standard widths and a range of heights, oftenexpressed in terms of “units”, but other types of rack are possible andrelevant for this problem. Again, in what follows, the term “rack” willbe used to include all of these possibilities, as well as any entitythat can hold one or more boxes and may need placement itself.

[0006] Similarly, a link can be any connection between two boxes,including a computer network link such as a copper, optical fibre,laser, or wireless link, which can itself be used as an Ethernet,FibreChannel, InfiniBand, telephone, wide-area, local-area, campus-area,metropolitan-area, serial, parallel, or other link type. Links caninclude other types of connections, too, such as pipes (e.g., forcooling fluids, hydraulic lines, or compressed air), cables (formechanical effects), and so on. To simplify exposition, this documentuses computer network components and devices as an exemplar problemdomain, but this in no way limits the scope of what is described herein.

[0007] One current solution to designing the racking and wiringconfigurations for networked devices is by manually designing theconfigurations. In designing the configurations for storage area network(SAN) devices, the above solution can sometimes be facilitated with theuse of visualization software such as, for example, variouscomputer-graphics drawing programs. However, this manual-based solutioncan be extremely time-consuming, suboptimal and error-prone forrealistic-sized SANs.

[0008] Another current solution to designing the racking and wiringconfigurations is to use a canned solution structure, such as the “groupcommon components together” approach. However, this solution is alsoerror-prone and typically results in designs that are more expensivethan necessary.

[0009] Another current solution in designing the racking and wiringconfigurations is to use existing algorithms for the bin-packingproblem, which will ensure that the boxes (e.g., SAN devices, computers,hubs, switches, and/or the like) are loaded into the minimum number ofracks. However, this solution disadvantageously disregards the cost andvulnerability of inter-rack wiring (i.e., wiring links that span betweenracks).

[0010] Thus, the current approaches and/or technologies are limited toparticular capabilities and/or suffer from various constraints.

SUMMARY OF EMBODIMENTS OF THE INVENTION

[0011] At least some of the various embodiments are now described. Inone embodiment of the invention, a method of designing the rackingconfiguration for boxes in racks and for determining which connectionsgo between different racks, includes:

[0012] solving a rack select optimization sub-problem to determine racksto use; and

[0013] solving a rack assign optimization sub-problem to determine whichparticular rack will hold a particular box.

[0014] In another embodiment, an apparatus for designing the rackingconfiguration for boxes in racks and for determining which connectionsgo between different racks, includes:

[0015] a machine-readable representation for a racking configurationproblem; and

[0016] a solver that can read that machine-readable representation andthat is configured to: solve a rack select optimization sub-problem todetermine at least one rack to use; and solve a rack assign optimizationsub-problem to determine which particular rack will hold at least onebox.

[0017] Other embodiments of the invention include, but are not limitedto, the various embodiments described below.

[0018] These and other features of an embodiment of the presentinvention will be readily apparent to persons of ordinary skill in theart upon reading the entirety of this disclosure, which includes theaccompanying drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019]FIG. 1 is a block diagram that illustrates a relationship betweenan optimization problem description and a solver, in accordance with anembodiment of the invention.

[0020]FIG. 2 is a flowchart of a method of designing the racking andwiring configuration for pieces of hardware (e.g., networked devices) bysolving optimization sub-problems, in accordance with an embodiment ofthe invention.

[0021]FIG. 3 is a block diagram that illustrates an example of solvingthe rack select optimization sub-problem, in accordance with anembodiment of the invention.

[0022]FIG. 4 is a block diagram that illustrates an example of solvingthe rack assign optimization sub-problem, in accordance with anembodiment of the invention.

[0023]FIG. 5 is a block diagram that illustrates an example of solvingthe rack layout optimization sub-problem, in accordance with anembodiment of the invention.

[0024]FIG. 6 is a block diagram that illustrates an example of using abin-packing bound to speed up the solution calculation, in accordancewith an embodiment of the invention.

[0025]FIG. 7 is a flowchart of method of forcing specifications in therack select optimization sub-problem, in accordance with an embodimentof the invention.

[0026]FIG. 8 is a block diagram of a method to allow the variables inthe quadratic variables Z( ) to be continuous, in accordance with anembodiment of the rack assign optimization method of the invention.

[0027]FIG. 9 is a flowchart of a method of using an anti-symmetryconstraint, in accordance with an embodiment of the invention.

[0028]FIG. 10 is a flowchart of using an initial solution from the rackselect optimization sub-problem for the rack assign optimizationsub-problem, in accordance with an embodiment of the invention.

[0029]FIG. 11 is a diagram of a method of using constraint branching, inaccordance with an embodiment of the invention.

[0030]FIG. 12 is a diagram of a method of using the number of boxesassigned to each rack in the rack-select optimization sub-problem as aguideline in the rack-assign optimization sub-problem.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0031] In the description herein, numerous specific details areprovided, such as examples of components and/or methods, to provide athorough understanding of embodiments of the invention. One skilled inthe relevant art will recognize, however, that an embodiment of theinvention can be practiced without one or more of the specific details,or with other apparatus, systems, methods, components, materials, parts,and/or the like. In other instances, well-known structures, materials,or operations are not shown or described in detail to avoid obscuringaspects of embodiments the invention.

[0032] A networked system may include “boxes” (i.e., pieces of hardwareor components) that need to be linked to each other to permitcommunication or functionality between the boxes. In an embodiment, thepresent invention provides a method and apparatus to automaticallydesign the racking and wiring configuration for boxes in the networkedsystem. The boxes and connections (links) between the boxes may beselected manually, or by use of a design tool, or by any other method.For example, components for a SAN may be selected by use of a SAN fabricdesign tool such as the SAN Configuration Tool from DELL COMPUTERCORPORATION, Round Rock, Tex.; the SAN Designer from TRUESAN NETWORKS,INC., San Jose, Calif.; or the Appia design tool described in “Appia:automatic storage area network design”, by Julie Ward, MichaelO'Sullivan, Troy Shahoumian, and John Wilkes (published in Conference onFile and Storage Technology (FAST' 02), pp. 203-217 (28-30 Jan. 2002,Monterey, Calif.) by USENIX, Berkeley, Calif.), which is hereby fullyincorporated herein by reference.

[0033] The boxes are placed in structures called “racks”.

[0034] In the racking and wiring configuration design problem, thefollowing sub-problems are typically considered:

[0035] (1) How many of each available rack type should be used forholding the selected boxes?

[0036] (2) In which rack should each box be placed?

[0037] (3) Where in each rack should each box be placed?

[0038] (4) What should be the layout of the loaded racks?

[0039] That is, in which physical location should each rack be placed?

[0040] Typically, the design objective(s) is at least one of thefollowing: to minimize the cost of the racks, to minimize the number ofused racks, to minimize the floor space or volume used by the racks (orthe cost of same), to minimize the electrical, heating, or cooling loadimbalance, and to minimize the wiring between racks (i.e., the number,cost and/or length of wires between racks). Many other possibleobjectives will be apparent to those skilled in the art. The constraintstypically imposed on the design problem may include at least one of theconstraints on the dimensions of the racks, power and coolingrequirements, presence and/or quantity of uninterruptable power supply(UPS), cable length, and other conditions that the user may specify inthe design problem. Again, many other constraints will be apparent tothose skilled in the art, and these should be taken only as exemplaryconstraints and objectives.

[0041] In an embodiment, the invention provides a method that can beexecuted by a computer. As a result, the method can solve much largerdesign problems than a human can solve by hand and can typically produceerror-free designs that are usually more cost-effective as compared todesigns produced by current approaches. As compared to existing methods,embodiments of the invention can model the special features of theracking and wiring problem more accurately and thus can produce moreeffective designs.

[0042] As described in detail below, an embodiment of the inventionprovides an optimization-based approach to the racking and wiring ofpieces of hardware such as, for example, networked boxes or otherhardware that were mentioned above. The embodiment may have versatilityfor application to a large variety of computer network (or othernetwork) design problems. In one embodiment, a method for designing theracking and wiring configuration for pieces of hardware includesseparating the design problem into four optimization sub-problems forrack selection, rack assignment, box placement, and rack layout. Themethod may provide flexibility to the user for guiding the solution, byallowing the user to adapt the optimization objective and constraints totheir specific concerns. The user may choose to alter the objectivedepending on which concerns are the most important (where the concernsmay include, for example, minimizing rack cost, saving floor space,preventing overheating by balancing power requirements, keeping theracks uncluttered to facilitate maintenance, and/or other concerns).

[0043] It is noted that embodiments of the invention are not limited todesigns of storage area networks.

[0044]FIG. 1 is a block diagram that illustrates a relationship betweenthe rack configuration design problem 105, an encoding 108 of theproblem in a language that a solver 110 can solve, to produce a solution120. Additionally, the solver may be fed control inputs or otherparameters 112 that can affect its execution. A graphical user interface(GUI) 102 (or other suitable interface) may be used to input variousparameters such as the parameters for defining the design problem 105.The solution 120 may be provided to the user via GUI 102. In anembodiment, the present invention provides an optimization-based methodfor designing the racking and wiring configuration for pieces ofhardware (such as networked devices) by expressing the racking andwiring configuration problem in an encoding 108 that uses a modelinglanguage that an integer programming solver 110 can exploit to produce asolution 120. The method is formulated in a unique way using integerprogramming techniques and various methods may be used to speed up thecalculation of the results, as described in detail below.

[0045] Integer programming techniques are described further in, forexample, the following reference: Integer Programming by Laurence A.Wolsey published by John Wiley and Sons, Inc. New York, 1998, which ishereby fully incorporated herein by reference. The encoding of theproblem model 105 and/or its solution method may be written in, forexample, GAMS (Generalized Algebraic Modeling System) which is ahigh-level modeling system for mathematical programming problems. GAMSreads in problems specified in the GAMS language and communicates withthe solver. The solver 110 may be CPLEX which is a collection ofmathematical programming software solvers available from ILOG, Inc.

[0046] As known to those skilled in the art, in integer programming,values are chosen for variables in order to maximize or minimize anobjective function, where the function is subject to constraints thatare expressed mathematically. The various constraints and objectivesmentioned in various embodiments of the invention may be selected orproduced by use of integer programming.

[0047] Method of Designing Racking and Wiring Configurations

[0048] In one embodiment, a method of the invention will consider a setof boxes and wiring requirements (i.e., which boxes are connected). Theset of boxes, racking and wiring requirements and constraints may, forexample, include at least some of the following:

[0049] (1) Set of required boxes.

[0050] (2) Set of required links between boxes.

[0051] (3) Set of available rack types.

[0052] (4) Set of floor locations for racks.

[0053] (5) Attributes for each box (e.g., dimensions, powerrequirements, UPS requirement, cooling requirements).

[0054] (6) Attributes of each rack (e.g., dimensions, power capacity,UPS availability or capacity, cost, cooling capacity, material, boundson link count, area, or volume that may enter or leave the rack).

[0055] (7) Attributes of each link (e.g., type, limits on length, area,weight, capacity, power, bending radius).

[0056] (8) Distances between floor locations.

[0057] (9) Other box, rack, or place-specific attributes andrequirements. (Many such requirements and attributes will be apparent tothose skilled in the art.)

[0058] Problem-specific constraints may also be specified, including,for example, one or more of the following:

[0059] (1) Pre-assignments (i.e., specifications of desired, existing,required or forbidden assignments of, e.g., boxes to racks or boxes thatshould be in the same rack).

[0060] (2) Maximum or minimum floor loadings or both.

[0061] (3) Maximum or minimum rack densities or both.

[0062] (4) A maximum total cost, floor space, and/or volume.

[0063] (5) A maximum total cable length.

[0064] (6) A maximum number or numbers of inter-rack links (possiblydifferent limits associated with different kinds of such links).

[0065] (7) A maximum length of, or cost of, inter-rack links.

[0066] (8) Maximum cooling loads, perhaps varying across the placeswhere racks may be put.

[0067] (9) Maximum or minimum distances between racks or both.

[0068] (10) Other problem-specific constraints. (Many such constraintswill be apparent to those skilled in the art.)

[0069] In an embodiment, a method of the invention will then provide atleast one of the following results.

[0070] (1) The number of instances of each available rack type which isused.

[0071] (2) The rack to which at least one box is assigned.

[0072] (3) The location to which at least one rack is assigned.

[0073] (4) The location in a rack to which at least some boxes areplaced or assigned.

[0074] At least one of the following objectives (subject to constraints)may be chosen, when solving for the above results.

[0075] (1) Minimize the number of racks to be used.

[0076] (2) Minimize the total cost of racks to be used.

[0077] (3) Minimize the cost of remaining racks to be purchased, in thecase where present racks can be used.

[0078] (4) Minimize the cost, length or number of inter-rack wires orlinks.

[0079] (5) Minimize the number of “long cables”, where a long cableexceeds a pre-defined length.

[0080] (6) Minimize the height of the center of gravity of a rack.

[0081] (7) Leave space for future growth, and slots for new boxes.

[0082] (8) Maximize the number of intra-rack wires or links.

[0083] (10) Other metric(s) or criteria chosen by the user.

[0084] It is noted that some of the above objectives may be desirablebecause racks tend to be expensive and/or may occupy expensive orlimited area in a room. Furthermore, racks may be placed in a room thatis being retrofitted, and may be constrained on the available power orcooling. Additionally, some boxes and racks may have particularrequirements such as uninterruptible power or specific cooling needs ormay need to be in a rack with earthquake resistance features.

[0085] Additionally, some of the above constraints may be desirablebecause: (1) the total height or space of boxes can not exceed thelimited total height or space of racks, (2) boxes that requireuninterruptible power must typically be in a rack that supports adequateuninterruptible power, (3) racks must typically meet the powerrequirements of their boxes, (4) the total heat generated by boxes in asingle rack must be within some limits (e.g., the cooling ability of therack in a particular configuration).

[0086] Additionally, some of the above objectives may be desirablebecause it is typically desirable to minimize the cables that connectboxes in different racks and to minimize in particular the number oflong cables that connect boxes in different racks.

[0087]FIG. 2 is a flowchart that illustrates a method 200 of designingthe racking and wiring configuration for pieces of hardware or boxes(such as networked devices), in accordance with an embodiment of theinvention. The method 200 permits the above results to be generatedbased upon a given set of inputs by solving various optimizationsub-problems. The method 200 may include encoding the problemspecification 105 in a form 108 that can be used by solver 110 (FIG. 1)in step 203. The method 200 may include computing (202) the upper boundson a number of rack instances required for each rack type, e.g., byusing a multi-dimensional bin-packing model, or other method, such asdesign by hand. The method 200 further includes solving (205) anoptimization sub-problem (i.e., “rack select”) to determine theparticular racks to use. The method 200 then solves (210) anoptimization sub-problem (i.e., “rack assign”) to determine whichparticular rack that each box should be assigned to. The method 200 thensolves (215) an optimization sub-problem (i.e., “rack layout”) todetermine which floor location that each loaded rack will be assignedto. Thus, in the rack layout model, the racks are treated as if theywere boxes, and the floor positions are treated as if they were racks.The above optimization sub-problems are typically solved in sequence toimprove execution time, in an embodiment of the invention. However, inother embodiments, any one these models may be solved individually(alone) rather than sequentially. For example, one embodiment mayinclude solving only the rack select optimization sub-problem and usingother methods to assign boxes to racks or to assign racks to physicallocations. As another example, a user can use a solution of the rackselect optimization sub-problem alone, if the user only wants to knowwhich particular racks to obtain or buy. Similarly, either of the otheroptimization sub-problems (rack assign optimization sub-problem or racklayout optimization sub-problem) may be solved alone. As mentionedabove, these optimization sub-problems may be represented as integerprogramming models, which may be solved using integer programmingsolvers. The integer program to solve the various above-mentionedsub-problems may be specified via an integer programming language.

[0088] An optional local improvement heuristic may then be selectivelyperformed (220) on the results from the above actions (205) to (215), inorder to improve the results further.

[0089] Optionally, an assignment can be made for the placement of boxeswithin a rack, or placement of at least one box within a rack. In FIG.2, this optional step is denoted as the “box placement” optimizationsub-problem. For example, the order of boxes within a rack can be madeto achieve an objective(s) such as, for example, minimizing cablelengths and/or achieving another optimization(s). Other objectives mayalternatively or additionally include, for example, weight distributionor power distribution. Other objectives may be selected.

[0090] As shown in FIG. 2, a “box placement” optimization sub-problemmay be optionally solved (212) after solving the “rack assign”optimization sub-problem. Alternatively or additionally, a “boxplacement” optimization sub-problem may be optionally solved (217) aftersolving the “rack layout” optimization sub-problem. The algorithm forplacing boxes in the “box placement” optimization sub-problem is similarto the below-described algorithm for placing racks within a room.

[0091] Rack Select Optimization Sub-Problem

[0092] Referring now to FIG. 3, there is shown a block diagram thatillustrates an example of solving the rack select optimizationsub-problem, in accordance with an embodiment of the invention. Thismodel chooses the set of racks such that all boxes fit in the chosenracks.

[0093] The constraint(s) used, for example, may include at least one ofthe following: (1) the total height of the boxes (i.e., pieces ofhardware) to be placed in a rack does not exceed a rack height; (2) aselected rack meets power requirements for boxes placed in the selectedrack; (3) boxes that require UPS are placed in a rack that supportsadequate UPS; (4) the total heat generated by boxes in a single rack arewithin some limits; (5) rack capacity are met with respect to othermeasured attributes of the boxes (besides power, UPS, cooling, and sizeattributes); (6) certain box type and rack type combinations may beprohibited; (7) a box and a backup of the box are placed in separateracks; (8) a box and a backup of the box are separated by at least someuser-defined distance; and (9) a box is assigned to a compatible rack.Other alternative or additional constraints may be selected. A backup ofa box is defined as a box that is meant to be the backup for others andthat if the original box fails, then the backup box is meant to takeover the function of the failing box. The above constraints can beselectively designated in the problem representation 108 (see FIG. 1),may be defined by an objective function in the solver 110, or providedas a control parameter or parameters to the solver 112, or somecombination of these.

[0094] The objective may be, for example, one of the following: (1) tominimize the total cost of selected racks; (2) to minimize the totalnumber of selected racks; (3) to minimize the cost of racks not yetpurchased (or not yet owned); (4) to minimize floor space needs; (5) tominimize power requirements; (6) to balance cooling requirements; and/or(7) to use a single rack type. Other alternatives or additionalobjectives may be selected. The objectives may be input in the encodedproblem specification 108 (FIG. 1), may be defined by an objectivefunction in the solver 110, or provided as a control parameter orparameters to the solver 112, or some combination of these.

[0095] In the example of FIG. 3, assume that there are two types (rt) ofpossible racks to choose from, namely a big-type rack (generallyrepresented by rack 300 where rt=big) and a small-type rack (generallyrepresented by rack 305 where rt=small). It is noted that other types(rt) of possible racks may be defined, such as, for example,blue-colored or other colored racks, wood racks, metal racks, differentbrand racks, and the like. The rack width may also be defined, althoughrack width tends to be standard. Furthermore, some types of possibleracks may be differ in cost, as well as in their limitations on powersupply capability, UPS capability, cooling capability, and/or otherphysical attributes. The different rack types may be input in theencoded problem specification 108 (FIG. 1), may be pre-defined in thesolver 110, or provided as a control parameter or parameters to thesolver 112, or some combination of these.

[0096] Assume that in the example of FIG. 3, the following racks (r) maybe selected as shown in Table 1. The number and type of selectable racksr may vary in other examples. TABLE 1 Rack r rack type (rt) r = big1(also referenced as rt = big rack 300a) r = big2 (also referenced as rt= big rack 300b) r = small1 (also referenced as rt = small rack 305a) r= small2 (also referenced as rt = small rack 305b) r = small3 (alsoreferenced as rt = small rack 305c)

[0097] Assume further that a box type is defined as “bt” and that thereare type A, B, C, and D boxes. Box type bt can be, for example, relatedto physical attribute(s) such as functionality (i.e., switch, hub, diskarray, server, and/or the like), size, shape, color, power requirement,cooling requirement, UPS requirement, and/or other physical attributes.Assume further that the following boxes have the corresponding box typebt as shown in Table 2. The number of boxes and box types bt may vary inother examples. TABLE 2 Box (b) Box type (bt) 310 A 315 A 320 B 325 C330 C

[0098] Assume the following decision variables are used in the solutionapproach for the rack select optimization sub-problem, as shown inequations (1a), (1b), and (1c):

[0099] Decision Variables

Y(r)=1 if rack r is used to hold one or more boxes  (Eq. 1a),

Y(r)=0 if rack r is not used to hold a box  (Eq. 1b),

NX(bt,r)=number of boxes of type bt placed in rack r  (Eq. 1c),

[0100] where Y is a binary variable and NX is a general integer variablewith upper bounds pre-computed based on the measurement attributes of btand rt and on the number of available boxes of type bt, whichever ismost binding.

[0101] Table 3 lists one possible (though not unique) solution forEquations (1a) to (1c) where the results from the decision variables inthe integer program indicate that rack big1 (300 a) will hold two type Aboxes (i.e., boxes 310 and 315), one type B box (i.e., box 320), and atype C box (may be box 325 or box 330), that rack small1 will hold atype C box (may be box 325 or box 330), and that rack big2, rack small2,and rack small3 will be unused (will not hold any of the boxes). Theresults are obtained based on the constraint(s) and objective(s) thatare specified, for example, in the problem representation 108 (FIG. 1).Thus, the results may vary, depending on the particular selectedconstraint(s) and objective(s). It is further noted that that the solverfor the rack assign optimization sub-problem (discussed in detail below)can determine which particular type C box (box 325 or box 330) will beplaced into rack big1 and into rack big2. TABLE 3 Results correspondingto the example of FIG. 3 Y(r) = 1 if rack r is used to hold one or moreboxes (Eq. 1a). Y(r) = 0 if rack r is not used to hold a box (Eq. 1b).Y(big1) = 1, since rack big1 will hold the various boxes (two type Aboxes, one type B box, and one type C box) in the example of FIG. 3;Y(big2) = 0, since rack big1 will not hold a box in the example of FIG.3; Y(small1) = 1, since rack small1 will hold a type C box in theexample of FIG. 3; Y(small2) = 0, since rack small2 will not hold a boxin the example of FIG. 3; Y(small3) = 0, since rack small3 will not holda box in the example of FIG. 3; NX(bt, r) = # of boxes of type bt placedin rack r (eq. 1c). NX(A, big1) = 2, since rack big1 will hold two typeA boxes (i.e., boxes 310 and 315) in the example of FIG. 3; NX(B, big1)= 1, since rack big1 will hold one type B box (i.e., box 320) in theexample of FIG. 3; NX(C, big1) = 1, since rack big1 will hold one type Cbox (i.e., may be box 325 or box 330) in the example of FIG. 3; NX(C,small1) = 1, since rack small1 will hold one type C box (i.e., may bebox 325 or box 330) in the example of FIG. 3; NX(A, big2) = 0, sincerack big2 will not hold a type A box in the example of FIG. 3; NX(B,big2) = 0, since rack big2 will not hold a type B box in the example ofFIG. 3; NX(C, big2) = 0, since rack big2 will not hold a type C box inthe example of FIG. 3; NX(A, small1) = 0, since rack small1 will nothold a type A box in the example of FIG. 3; NX(B, small1) = 0, sincerack small1 will not hold a type B box in the example of FIG. 3; NX(A,small2) = 0, since rack small2 will not hold a type A box in the exampleof FIG. 3; NX(B, small2) = 0, since rack small2 will not hold a type Bbox in the example of FIG. 3; NX(C, small2) = 0, since rack small2 willnot hold a type C box in the example of FIG. 3; NX(A, small3) = 0, sincerack small3 will not hold a type A box in the example of FIG. 3; NX(B,small3) = 0, since rack small3 will not hold a type B box in the exampleof FIG. 3; NX(C, small3) = 0, since rack small3 will not hold a type Cbox in the example of FIG. 3.

[0102] Table 4 lists example mathematical expressions for the rackselect optimization sub-problem, where the selected objective is tominimize the cost of selected racks. It is noted that these expressionsare not limiting to the scope of embodiments of the invention and thatthe mathematical expressions may differ based on the selectedobjective(s) and/or selected constraint(s). TABLE 4 MathematicalExpressions for the Rack Select optimization sub-problem Let [k] denotethe set {1, . . ., k} throughout. For Rack Select, the inputs are   SetRT of rack types.   Set BT of box types.   Set ATT of attributes of boxtypes.   Parameters cap(rt, att) and req(bt, att) for capacities andrequirements of each bt ∈ BT and rt ∈ RT for an attribute att ∈ ATT.  Cost cost(rt) of each rack type rt.   The maximum number of instancesneeded for each rack type MN(rt).   Number of boxes of each type NB(bt).Decision variables are   Indicator variable for whether each rack isused, Y(rt, rn), rt ∈ RT, rn ∈ [MN(rt)]. The variable Y(rt, rn) equals 1when at least one component is placed in instance rn of rack type rt andequals 0 otherwise.   The number of boxes assigned to each rack NBA (bt,rt, rn), bt ∈ BT, rt ∈ RT, rn ∈ [MN(rt)]. This must be a non- negativeinteger. If we are minimizing the cost of racks used the objective is  Minimize Σ  cost(rt) * Y(rt, rn)     rt∈RT, rn∈[MN(rt)] Otherobjectives are possible. A preferred embodiment of the invention usesthe following constraints. Some are optional, while similar constraintscan be added while remaining in the scope of embodiments of theinvention.   Assign each box to some rack,       Σ    NBA(bt, rt, rn) =NB(bt), bt ∈ BT.     rt∈RT, rn∈[MN(rt)]   where rt represents the racktype and rn represents the index of an instance of that rack type.   Donot use any rack unless it is paid for,       NBA(bt, rt, rn) ≦ Y(rt,rn) * NB(bt),   for bt ∈ BT, rt ∈ RT, rn ∈ [MN(rt)].   Obey rackcapacities,       Σ  NBA(bt, rt, rn) * req(bt, att) ≦ cap(rt)      bt∈BT   for att ∈ ATT, rt ∈ RT, rn ∈ [MN(rt)].   Anti-symmetryconstraints on racks,         Y(rt, rn + 1) ≧ Y(rt, rn),   rt ∈ RT, rn ∈[MN(rt) − 1].   One can forbid certain box types from appearing in givenrack types. The constraint would be     NBA(bt, rt, rn) = 0, rn ∈[MN(rt)] for given pairs (bt, rt) with bt ∈ BT and rt ∈ RT.

[0103] Rack Assign Optimization Sub-Problem

[0104] As noted in the example of FIG. 3, the rack big1 (300 a) willhold a type C box (may be box 325 or box 330), and the rack small1 willhold a type C box (may be box 325 or box 330). The rack assignoptimization sub-problem is that of determining to which rack a box willbe assigned, preferably for each box in the problem specification. Thus,the rack assign optimization sub-problem solver can determine whichparticular type C box (box 325 or box 330) will be placed into rack big1and into rack big2 in the above example.

[0105] The rack assign optimization sub-problem solver will typicallyconsider constraints and objectives such as, for example, at least oneof the following selectable constraint(s) and objective(s): (1) minimizethe total wire or link length; (2) minimize the total number ofinter-rack links or wires; (3) minimize the number of wires or linksexceeding a defined length; (4) minimize the total length of inter-rackwires or links; (5) minimize the cost of inter-rack wires or links; (6)minimize the number of wires or links crossing between machine rooms,domains, and/or buildings; (6) minimize the height of the center ofgravity of a rack; (7) leave space for future growth, and slots for newboxes; (8) maximize the number of intra-rack wires or links; (9) assigneach box to a rack; and/or (10) other selectable constraints.

[0106] Additionally, other constraints may be selected, such as,requiring a particular box b to be placed in a particular rack r, orrequiring that a particular box b can not be placed in a particular rackr. These different constraint(s) and objective(s) may be input in theencoded problem specification 108 (FIG. 1), may be pre-defined in thesolver 110, or provided as a control parameter or parameters to thesolver 112, or some combination of these. These are forced decisionsthat the user can specify, in order to speed up the integer programcalculation or to ensure the user's preferences are honored or both.

[0107] It is noted that the problem specification 105 includes(implicitly or explicitly) the physical attributes of a box type bt.Each type C box has the same physical attributes. As shown in FIG. 4,the type C boxes 325 and 330 will have the same physical attributes.

[0108] The problem specification 105 and its encoding 108 also includes(implicitly or explicitly) the individual box wiring layoutrequirements. For example, type C box 330 may require more wiringconnection than type C box 325, depending upon, for example, the wiringlayout or the function to be performed by the box.

[0109] Assume that the following decision variables are used in the rackassign optimization sub-problem solver, as shown in equations (2a),(2b), and (2c): $\begin{matrix}{\underset{\_}{{Decision}\quad {Variables}}{{{X\left( {b,r} \right)} = {1\quad {if}\quad {box}\quad b\quad {is}\quad {assigned}\quad {to}\quad {rack}\quad r}},}} & \left( {{{Eq}.\quad 2}a} \right) \\{{{X\left( {b,r} \right)} = {0\quad {if}\quad {box}\quad b\quad {is}\quad {not}\quad {assigned}\quad {to}\quad {rack}\quad r}},} & \left( {{{Eq}.\quad 2}b} \right) \\{{{{Z\left( {a,r,b,s} \right)} = {1\quad {if}\quad {box}\quad a\quad {is}\quad {assigned}\quad {to}\quad {rack}\quad r}}\quad {AND}\quad {{{box}\quad b\quad {is}\quad {assigned}\quad {to}\quad {rack}\quad s} = {{quadratic}\quad {term}\quad {X\left( {a,r} \right)}*{X\left( {b,s} \right)}\quad \left( {r \neq s} \right)}}},} & \left( {{{Eq}.\quad 2}c} \right)\end{matrix}$

[0110] and where X is a binary variable, and where Z can take on valuesof zero (0) or one (1). As discussed below, Z can be treated as acontinuous variable in order to speed up the calculation process. Also,Z(a,r,b,s) is defined only if components a and b are linked.

[0111] Table 5 lists the results for Equations (2a) to (2c) where theresults from the decision variables in the integer program indicate thatrack big1 (300 a) will hold box 310, box 315, box 320), and the type Cbox 330, and that rack small1 will hold the type C box 325. The resultsare obtained based on the constraint(s) and objective(s) that areselected for execution in the solver 110 (FIG. 1). Thus, the results mayvary, depending on the particular selected constraint(s) andobjective(s). TABLE 5 Results corresponding to the example of FIG. 4X(b, r) = 1 if box b is assigned to rack r (Eq. 2a), X(b, r) = 0 if boxb is not assigned to rack r (Eq. 2b), X(310, big1) = 1 X(310, big2) = 0X(310, small1) = 0 X(310, small2) = 0 X(310, small3) = 0 X(315, big1) =1 . . . X(320, big1) = 1 . . . other values for the above equations willbe zero (0). Z(a, r, b, s) = 1 if box a is assigned to rack r AND box bis assigned to rack s = quadratic term X(a, r) * X(b, s) (r ≠ s) (Eq.2c), Others will be one. For example Z(310, big1, 315, big1) = 1. Z(330,big1, 325, small1) = 1 Z(330, big1, 310, small1) = 0 . . . other valuesfor the above equation will be zero (0).

[0112] It is further noted that in the example of FIG. 4, typically dataindicating existing link instances are also considered in the quadraticterm Z( ). The number of existing link instances can vary. Thus, theZ(a,r,b,s) variables exist only when link(a,b)>0 and r<>s. Thus, forpurposes of describing a functionality of embodiments of the invention,the example of FIG. 4 only shows a subset of existing variables that areconsidered in the quadratic term Z( ).

[0113] Table 6 lists examples of mathematical expressions for the rackassign optimization sub-problem, where the selected objective is tominimize the cost of inter-rack wires. It is noted that theseexpressions are not limited to the scope of embodiments of the inventionand that the mathematical expressions may differ based on the selectedobjective(s) and/or selected constraint(s). TABLE 6 MathematicalExpressions for the rack assign optimization sub- problem For RackAssign, the inputs are   The inputs to Rack Select.   The NBA valueschosen in Rack Select.   Set of box instances BI.   A distance betweenrack instances dist(rt₁, rn₁, rt₂, rn₂) where rt₁, rt₂ ∈ RT, rn₁ ∈[MN(rt₁)] and rn₂ ∈ [MN(rt₂)].   The number of cables cables(bi₁, bi₂)between two box instances bi₁, bi₂ ∈ BI.   Indicator variables force(bi,rt, rn) and forbid(bi, rt, rn) which are 1 when the box instance is birequired or forbidden to be in rack(rt, rn), rt ∈ RT, rn ∈ [MN(rt)]. ForRack Assign, the outputs are   An indicator variable X(bi, rt, rn) whichis 1 when box bi is assigned to rack(rt, rn) and is 0 otherwise, bi ∈BI, rt ∈ RT, rn ∈ [MN(rt)].   An indicator variable Z(bi₁, rt₁, rn₁,bi₂, rt₂, rn₂) which is 1 when box bi₁ is assigned to rack(rt₁, rn₁) andbox bi₂ is assigned to rack(rt₂, rn₂) and is 0 otherwise, bi₁, bi₂ ∈ BI,rt₁, rt₂ ∈ RT, rn₁ ∈ [MN(rt₁)] and rn₂ ∈ [MN(rt₂)]. An example objectiveis   Minimize   Σ    cables(bi₁, bi₂) * dist(rt₁, rn₁, rt₂, rn₂) bi₁,bi₂∈BI, rt₁, rt₂∈RT * Z(bi₁, rt₁, rn₁, bi₂, rt₂, rn₂) rn₁∈[MN(rt₁)],rn₂∈[MN(rt₂)] which minimizes the total length of inter-rack cables. Apreferred embodiment uses the following constraints. Some are optional,while similar constraints can be added while remaining within the scopeof embodiments of this invention.   Only use racks used in the solutionof Rack Select:     X(bi, rt, rn) = 0   for all bi ∈ BI if Y(rt, rn) = 0in Rack Select.   Rack capacities need to be respected:   Σ  req(bi,att) * X(bi, rt, rn) ≦ cap(rt, att)   bi∈BI   for rt ∈ RT, rn ∈ [MN(rt)]and att ∈ ATT. The term req(bi, att) represents the amount of attributeatt used by box instance bi.   Use at least an α fraction of theresources of each rack used in Rack Select where α = 0.85 in oneembodiment:     Σ  req(bi, att) * X(bi, rt, rn)     bi∈BI       ≧     αΣ req(bt, att) * NBA(bt, rt, rn),     bt∈BT for rt ∈ RT, rn ∈ [MN(rt)]and att ∈ ATT. This constraint, like others, is optional and can beeither left out or made trivially satisfied by setting α = 0.   Observethe force and forbid constraints:     force(bi, rt, rn) = 1 $$ X(bi, rt,rn) = 1   and     forbid(bi, rt, rn) = 1 $$ X(bi, rt, rn) = 1   for bi ∈BI, rt ∈ RT, rn ∈ [MN(rt)].   Logic relating X and Z variables:   X(bi₁,rt₁, rn₁) + X(bi₂, rt₂, rn₂) ≦ Z(bi₁, rt₁, rn₁, bi₂, rt₂, rn₂) + 1   forbi₁, bi₂ ∈ BI, rt₁, rt₂ ∈ RT, rn₁ ∈ [MN(rt₁)] and rn₂ ∈ [MN(rt₂)].  Place “about” the same number of boxes into each rack as “Rack Select”did: [(1 − β)NBA(bt, rt, ri)] ≦ Σ X(bi, rt, rn) bi∈BI   ≦ [(1 +β)NBA(bt, rt, ri)]   for bt ∈ BT, rt ∈ RT and rn ∈ [MN(rt)]. The β is aparameter; β is equal to 0.2 in one embodiment of the invention.

[0114] Rack Layout Optimization Sub-Problem

[0115] The rack layout optimization sub-problem is to determine thefloor position and ordering of the racks r. In the solution model forthis sub-problem, the racks are treated as if they were boxes, and thefloor positions are treated as if they were racks. Thus, the rack layoutoptimization model uses the same approach as the rack assignoptimization model and in at least one embodiment is treated as a caseof the rack assign optimization sub-problem.

[0116] The rack layout optimization model will typically considerconstraints and objectives, such as, for example, at least one of thefollowing selectable constraint(s) and objective(s): (1) minimize thetotal link or wire length; (2) minimize the total number of inter-racklinks or wires; (3) minimize the number of links or wires exceeding adefined length; (4) minimize the total length of inter-rack links orwires; (5) minimize the cost of inter-rack links; (6) minimize thenumber of links or wires crossing between machine rooms, domains, and/orbuildings; (7) minimize floor space needs; (8) minimize powerrequirements; (9) to balance cooling requirements; and/or (10) otherselectable constraints. These different constraint(s) and objective(s)may be input in the encoded problem specification 108 (FIG. 1), may bepre-defined in the solver 110, or provided as a control parameter orparameters to the solver 112, or some combination of these.

[0117] Assume the following decision variables are used in the racklayout optimization model, as shown in equations (3a), (3b), and (3c):$\begin{matrix}{\underset{\_}{{Decision}\quad {Variables}}{{{X\left( {r,L} \right)} = {1\quad {if}\quad {rack}\quad r\quad {is}\quad {assigned}\quad {to}\quad {location}\quad L}},}} & \left( {{{Eq}.\quad 3}a} \right) \\{{{X\left( {r,L} \right)} = {0\quad {if}\quad {rack}\quad r\quad {is}\quad {not}\quad {assigned}\quad {to}\quad {location}\quad L}},} & \text{(Eq.~~3b)} \\{{{{Z\left( {r,L,s,M} \right)} = {1\quad {if}\quad {rack}\quad r\quad {is}\quad {assigned}\quad {to}\quad {location}\quad L}}\quad {AND}\quad {{{rack}\quad s\quad {is}\quad {assigned}\quad {to}\quad {location}\quad M} = {{quadratic}\quad {term}\quad {X\left( {r,L} \right)}*{X\left( {s,M} \right)}\quad \left( {L \neq M} \right)}}}\quad} & \left( {{{Eq}.\quad 3}c} \right)\end{matrix}$

[0118] and where X is a binary variable, and where Z can take on valuesof zero (0) or one (1). As discussed below, Z can be treated as acontinuous variable in order to speed up the calculation process.Z(r,L,s,M) should be defined only if the components r and s are linked.

[0119] Table 7 lists the results for Equations (3a) to (3c) where theresults from the decision variables in the integer program indicate thatrack 500 will be in location 530, rack 505 will be in location 535,racks 510 and 515 will be in location 540, and rack 520 will be inlocation 545. The results are obtained based on the constraint(s) andobjective that are selected for execution in the solver 110 (FIG. 1).Thus, the results may vary, depending on the particular selectedconstraint(s) and objective(s). TABLE 7 Results corresponding to theexample of FIG. 5 X(r, L) = 1 if rack r is assigned to location L (Eq.3a), X(r, L) = 0 if rack r is not assigned to location L (Eq. 3b),X(505, 535) = 1 X(500, 530) = 1 X(515, 540) = 1 X(520, 545) = 1 X(505,530) = 0 X(505, 540) = 0 . . . other values for the above equations willbe zero (0). Z(r, L, s, M) = quadratic term X(r, L) * X(s, M) (L ≠ M)(Eq. 3c), Z(515, 540, 510, 540) = 1 Z(515, 530, 510, 540) = 0 . . .other values for the above equation will be zero (0).

[0120] It is further noted that in the example of FIG. 5, typically dataindicating existing link instances are also considered in the quadraticterm Z( ). The number of existing link instances can vary. Thus, theZ(r,L,s,M) variables exist only when link(r,s)>0 and L<>M. Thus, forpurposes of describing a functionality of embodiments of the invention,the example of FIG. 5 only shows a subset of existing variables that areconsidered in the quadratic term Z( ).

[0121] The parameters r and s are boxes. Z(r,L,s,M) is defined only whenboxes r and s are connected by one or more links.

[0122] Perform a Final Local Search to Improve the Solution

[0123] Since the above optimization models are solved sequentially, aless than optimal result (solution) may possibly be obtained for theobjective than if the above models are solved concurrently. A finallocal search may be performed to improve the result (solution) for theobjective. This is a heuristic-based approach where, for example, a boxmay be moved to a different rack to determine if the objective improves.As another example, the locations of two boxes may be swapped (within arack or between racks) to determine if the objective improves. Asanother example, a box with many wiring connections may be moved to acentrally-located rack or to other racks. As another example, a rack maybe moved to another location to determine if the objective improves. Asanother example, a pair of racks may be swapped in locations todetermine if the objective improves. Other possible methods may be usedin this step of performing the final local search to determine a bettersolution that improves the objective. The method or methods to use canbe described in the encoded problem specification 108, embedded in theimplementation of solver 110, or provided to the solver 110 through thecontrol parameters 112, or some combination of these.

[0124] Methods for Limiting the Search Space to Speed Up the IntegerProgram Solution Calculation

[0125] (a) Use a Fraction of a Given Constraint (Method of Using aBin-Packing Bound)

[0126] A method to speed up the calculation for the solution in thesolver 110 (FIG. 1) is to use a multidimensional bin-packing heuristicmethod. By applying such a method, one can obtain two benefits: first,the bin-packing method provides an initial solution to the problem 105,and second, it provides bounds on the numbers of racks of each type thatwould be required by the rack select optimization model. Since integerprograms often require significant time in finding an initial feasiblesolution, using the bin-packing heuristic to generate an initialsolution may speed up the solver. In addition, having bounds on thenumbers of racks of each type reduces the size of the solution spacethat must be searched by the solver, and hence reduces the solutiontime. The bin-packing algorithm used can be described in the encodedproblem specification 108, embedded in the implementation of solver 110,or provided to the solver 110 through the control parameters 112, orsome combination of these.

[0127] One such multidimensional heuristic bin-packing method is afirst-fit bin-packing method. This method considers the boxes in anychosen order. It considers each box in sequence and assigns the box tothe first rack in which it fits with respect to all of the rack'sconstraints. For example, in FIG. 6 assume that rack 600 has a height of100 inches and power supply capability of pr=200 Watts. Assume that box620 has a height of 50 inches and power requirement pr=150 Watts. Assumethat box 625 has a height of 40 inches and power requirement pr=40Watts. Assume that box 627 has a height of 40 inches and powerrequirement pr=75 Watts. If box 620 is first placed in the rack 600,then box 627 will not be placed in the rack 600, since the powerrequirements of boxes 620 and 627 (pr=150W+75W=225W) exceeds the powersupply capability (pr=200W) of the rack 600. However, the box 625 can beplaced in the rack 600, since the power requirement of boxes 620 and 625(pr=190W=150W+40W) is less than the power supply capability of the rack600.

[0128] As another example, assume that the rack 605 has a temperaturereduction capability of 40 degrees. The box 630 can fit in the rack 605,since the box 630 has a cooling requirement of 40 degrees. On the otherhand, the box 635 will not fit in the rack 605, since the box 635 has acooling requirement of 30 degrees.

[0129] The order in which the boxes are considered can vary. In oneembodiment, one might order the boxes according to a particularattribute such as height or power consumption. In other embodiments,different orderings, such as a random ordering, or several differentrandom orderings, may be considered, and one of these selected forfurther consideration. The choice of order to use can be described inthe encoded problem specification 108, embedded in the implementation ofsolver 110, or provided to the solver 110 through the control parameters112, or some combination of these.

[0130] Thus, in an embodiment, a method to speed up the calculation forthe solution in the rack select optimization sub-problem includesfinding upper bounds on the required numbers of racks of each type. Thesolution of the integer program is sped up by considering only thosesolutions that adhere to the bounds. Thus, this method includescalculating bounds on the maximum number of each rack type to be used ina solution, and considering only those solutions adhering to the bounds.The upper bounds are found by performing bin-packing, or some othermethod, such as manual design. The upper bounds need not be tight upperbounds, although the tighter they are, the fewer alternative designswill need to be considered, and the faster the solver will run.

[0131] In another embodiment, the cost of the bin-packing solution isused to bound the integer program objective. In other words, solving therack select optimization sub-problem includes using a cost obtained fromthe bin-packing method.

[0132] (b) Force a Certain Assignment(s) (or Requirement(s)) to Speed Upthe Calculation

[0133] As mentioned above, the user can force a certain assignment(s),or require other specification requirement(s), in an attempt to speed upthe integer program calculation. In other words, this method involvesforcing of at least one requirement related to a box appearing in agiven rack instance (r) in order to increase the solution calculationspeed for the rack assign optimization sub-problem. These forcedassignments may include, but are not limited to, for example, requiringa particular box b to be placed in a particular rack r, requiring that aparticular box b can not be placed in a particular rack r, and/orrequiring a certain number and/or types of racks to be used. This forcedassignment may also be used to indicate the presence of existing,placed, mounted or assigned boxes and racks; such existing placementscan be provided in order to minimize the amount of physicalre-arrangements required by the solution, as well as in order to speedup the solver.

[0134] As shown in the flowchart in FIG. 7, a method 700 for forcing acertain assignment includes requiring (705) at least one specification(i.e., forcing of at least one requirement related to a box appearing ina given rack instance, such as forcing a certain assignment). Forexample, action (705) may include forcing a box assignment to aparticular rack. The method then solves (710) the rack selectoptimization sub-problem to determine the racks to use. The algorithmfor forcing a certain assignment(s) (or requirement(s)) to speed up thecalculation can be described in the encoded problem specification 108,embedded in the implementation of solver 110, or provided to the solver110 through the control parameters 112, or some combination of these.

[0135] (c) Allow the Variables in the Quadratic Term Z( ) to beContinuous

[0136] By permitting the variables in the quadratic term Z( ) (see aboveequations 2c and 3c) to be continuous, the integer program will be ableto speed up the calculation for the results. This method of relaxing theintegrality constraints by making the Z( ) variables be continuous isadvantageous for faster calculations.

[0137] As shown in the method 800 in FIG. 8, one of the constraints isZ(a,r,b,s)+1≧X(a,r)+X(b,s) (see action (805)). As a result, if both Xterms, X(a,r) and X(b,s), are 1 then Z is forced to be ≧1. In fact Zwill equal 1 if its range is [0, 1]. When at least one X is zero, Z cantake on any number in [0,1] (see action (810)). However, because this isa minimization program and all Zs have positive coefficient, Z will takeon the smallest possible value, 0, at the optimum. The algorithm forallowing continuous variables in the quadratic term Z( ) can bedescribed in the encoded problem specification 108, embedded in theimplementation of solver 110, or provided to the solver 110 through thecontrol parameters 112, or some combination of these.

[0138] (d) Use an Anti-Symmetry Constraint

[0139] As noted in equation (1a) above, Y(r)=1 if rack r is used to holda box. In an embodiment of the invention, a method of usinganti-symmetry constraints advantageously eliminates or reduces thenumber of identical solutions to the rack select optimizationsub-problem. In this context, term “symmetry” is defined as equal (ornearly equal) load or equal (or substantially equal) objective.Anti-symmetry constraints are used in integer programs to reduce a setof functionally equivalent, redundant solutions to a single solution.

[0140] As shown in the flowchart in FIG. 9, a method 900 for forcing ananti-symmetry constraint(s) includes forcing (905) an anti-symmetryconstraint. For example, by specifying Y(r)≧Y(r+1), then a particularrack instance r will be required to be used before designating for use asecond rack instance r+1. In action (905), prior to designating the rackr, all racks may be considered, all but a particular rack r=R may beconsidered, all but racks r=(R,R+1) may be considered, or othercombinations may be considered or not considered. The method 900 thensolves (910) the rack select optimization sub-problem to determine theracks to use. It is noted that for FIG. 9, the anti-symmetry constraintapplies only if the two racks in the constraint are of the same type. Wecan fix this either by using two indices (rack type, instance number) asin, for example, the GAMS code, or we can just add the phrase “if rack rand rack r+1 are of the same type”. In action (905), Y(r )>=Y(r+1)whenever racks r and r+1 are of the same type.

[0141] The addition of these constraints ensures that a logicaldistinction among identical racks, and attempts to reduce the set offunctionally equivalent solutions. For example, suppose we have theoption of using two racks of the same type. Obviously, given anassignment of boxes to racks, one can swap the contents of these tworacks and get another solution. All such solutions will be considered bythe solver. Therefore, constraints are added to cut down on the numberof equivalent solutions that will be considered.

[0142] The algorithm for using an anti-symmetry constraint can bedescribed in the encoded problem specification 108, embedded in theimplementation of solver 110, or provided to the solver 110 through thecontrol parameters 112, or some combination of these.

[0143] (e) Use the Initial Solution from the Rack Select Method for theRack Assign Method

[0144] An initial feasible integer solution to the rack assignoptimization sub-problem is to use the optimal solutions from the rackselect optimization sub-problem. Because integer programming solversoften spend a large portion of their solution time in finding a firstfeasible integer solution, this method of using the optimal solutionfrom the rack assign model may help speed up the solution calculationtime.

[0145] As shown in the flowchart in FIG. 10, a method 1000 of reducingthe calculation time in the rack assign optimization sub-problemincludes calculating (1005) the optimal solution(s), such as the Y(r)and NX(bt,r) solutions, in the rack select optimization sub-problem. Itis noted that the rack select optimization model typically can onlyprovide a partial solution to the rack assign optimization sub-problem,since the rack select optimization sub-problem is not aware of boxinstances and their connections. In some cases, it is also noted thatwhatever information the rack select optimization sub-problem providesto the rack assign optimization sub-problem is typically only forinitialization. However, the rack select optimization sub-problemprovides information for the rack assign optimization sub-problem theparticular racks to consider for box assignments. There are boxassignments that can be inferred from the solution to the rack selectoptimization sub-problem, and these assignments are used asinitialization for the rack assign optimization sub-problem but the rackselections from the rack select optimization sub-problem may bepermanent.

[0146] The method 1000 further includes inferring (1010) a feasibleassignment of box instances to the selected rack instances consistentwith the optimal solution of the Rack Select optimization sub-problem.Using this feasible assignment, the method 1000 constructs (1015) aninitial incumbent (best feasible) integer solution to the Rack Assignoptimization sub-problem. This will help speed up solution of the RackAssign sub-problem. The algorithm for using the initial solution fromthe rack select method for the rack assign method can be described inthe encoded problem specification 108, embedded in the implementation ofsolver 110, or provided to the solver 110 through the control parameters112, or some combination of these.

[0147] (f) Application of Constraint Branching

[0148]FIG. 11 is a flow diagram to illustrate a method 1100 applicationof constraint branching, in accordance with an embodiment of theinvention. In the rack assign optimization model, the followingauxiliary variables YKW(rt,r) may be used (1105). YKW(rt,r) representsthe number of boxes assigned to rack (rt,r). Thus, for example, YKW(b,r) is equal to the summation of X(b, r) over boxes b, while YKW(s, r) isequal to the summation of X(s, r) over boxes s. The constraints whichenforce this definition are nonbinding, accounting equations that serveto speed up the enumeration. The YKW variables are treated as integervariables and are given a high branching priority in the brand and boundenumeration (action (1110)). The branching priorities of the X variablesare lower than the branching priorities of the YKW variables (action1115). Within the X variables, higher priority is given to boxes withmore links (action 1120). Thus, constraint branching may be used tospeed up the calculation of the solution in the rack assign optimizationsub-problem.

[0149] Thus, an embodiment of the invention provides a method ofchoosing branching priorities in the rack assign optimizationsub-problem and selecting what particular variables receive priority.For additional details on constraint branching, see D. M. Ryan and B. A.Foster, “An Integer Programming Approach to Scheduling” in ComputerScheduling of Public Transport Urban Passenger Vehicle and CrewScheduling, edited by A. Wren, North-Holland Publishing Company, NorthHolland, Amsterdam (1981), pp. 269-280, which is hereby fullyincorporated herein by reference. The algorithm for applying constraintbranching can be described in the encoded problem specification 108,embedded in the implementation of solver 110, or provided to the solver110 through the control parameters 112, or some combination of these.

[0150] (g) Using the Number of Boxes Assigned to Each Rack in theRack-Select Phase as a Guideline in the Rack-Assign Phase.

[0151]FIG. 12 is a flow diagram of a method 1200 in accordance with anembodiment of the invention. In the rack select phase (rack selectoptimization sub-problem), the number of instances of each rack type ischosen (action (1205)). In the rack select optimization phase, thenumber of boxes of each type to assign to each rack instance is chosen(action (1210)). In the rack-assign optimization phase (rack assignoptimization sub-problem), we ensure that approximately the same numberof boxes of each type are assigned to each rack instance (action(1215)). This helps cut down the search space. It is noted that themethod 1200 is a heuristic for speeding up the solution calculation ofthe problem (at a potential loss of optimality).

[0152] For example, if ten (10) hosts are assigned to rack small1 in therack select space and we allow 20% variation, we would only considersolutions in the rack assign phase where between eight (8) and twelve(12) hosts are assigned to rack small1. The above algorithm can bedescribed in the encoded problem specification 108, embedded in theimplementation of solver 110, or provided to the solver 110 through thecontrol parameters 112, or some combination of these.

[0153] Meta-Constraints

[0154] It is noted that in the rack select optimization sub-problem, thebox constraints (such as size, power requirements, UPS requirement, andcooling requirement, or other various combinations of constraints) maybe defined by a general commodity or “meta-constraints”. A rack istypically assumed to provide a limited supply of the meta-constraint.Thus, the constraints specified in the integer program may be defined asa general commodity. Meta-constraint typically comprises the sum ofindividual constraints that were mentioned above. For example, we mightadd the meta-constraint “weight.” Each rack instance would have amaximum weight of components it can support. Each component type wouldhave a weight. Then the sum of the weights of all components assigned toa rack could not exceed the rack's weight limit. The algorithm fordefining a meta-constraint can be described in the encoded problemspecification 108, embedded in the implementation of solver 110, orprovided to the solver 110 through the control parameters 112, or somecombination of these.

[0155] Application to Networked Boxes without Wiring Requirements

[0156] It is noted that some boxes r may not require physical links. Forexample, boxes r that communicate via wireless links or a wirelessprotocol. Additionally, some boxes may have no connections; and some ofthese boxes may really be “holes” to reserve space for some reason(e.g., to meet cooling goals, or to provide empty spaces in order toallow for future growth). Some networks may also be implemented with theuse of boxes r that require a wired link between the boxes. Thus, in anembodiment of the invention, the wiring constraints noted above may beomitted (alternatively, not supplied as part of the encoded problemspecification 108) when solving the various optimization sub-problems orwhen using the heuristic to locally improve the solution or both. Thealgorithm for racking, wiring, and/or placement of boxes without wiringrequirements can be described in the encoded problem specification 108,embedded in the implementation of solver 110, or provided to the solver110 through the control parameters 112, or some combination of these.

[0157] The various engines or modules discussed herein may be, forexample, software, commands, data files, programs, code, modules,instructions, or the like, and may also include suitable mechanisms.

[0158] Reference throughout this specification to “one embodiment”, “anembodiment”, or “a specific embodiment” means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present invention. Thus,the appearances of the phrases “in one embodiment”, “in an embodiment”,or “in a specific embodiment” in various places throughout thisspecification are not necessarily all referring to the same embodiment.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments.

[0159] Other variations and modifications of the above-describedembodiments and methods are possible in light of the foregoingdisclosure. Further, at least some of the components of an embodiment ofthe invention may be implemented by using a programmed general purposedigital computer, by using application specific integrated circuits,programmable logic devices, or field programmable gate arrays, or byusing a network of interconnected components and circuits. Connectionsmay be wired, wireless, by modem, and the like.

[0160] It will also be appreciated that one or more of the elementsdepicted in the drawings/figures can also be implemented in a moreseparated or integrated manner, or even removed or rendered asinoperable in certain cases, as is useful in accordance with aparticular application.

[0161] It is also within the scope of an embodiment of the presentinvention to implement a program or code that can be stored in amachine-readable medium to permit a computer to perform any of themethods described above.

[0162] Additionally, the signal arrows in the drawings/Figures areconsidered as exemplary and are not limiting, unless otherwisespecifically noted. Furthermore, the term “or” as used in thisdisclosure is generally intended to mean “and/or” unless otherwiseindicated. Combinations of components or steps will also be consideredas being noted, where terminology is foreseen as rendering the abilityto separate or combine is unclear.

[0163] As used in the description herein and throughout the claims thatfollow, “a”, “an”, and “the” includes plural references unless thecontext clearly dictates otherwise. Also, as used in the descriptionherein and throughout the claims that follow, the meaning of “in”includes “in” and “on” unless the context clearly dictates otherwise.

[0164] It is also noted that the various functions, variables, or otherparameters shown in the drawings and discussed in the text have beengiven particular names (such as Y(r), NX(bt,r), X(b,r), Z(a,r,b,s), andthe like) for purposes of identification. However, the function names,variable names, or other parameter names are only provided as somepossible examples to identify the functions, variables, or otherparameters. Other function names, variable names, or parameter names maybe used to identify the functions, variables, or parameters shown in thedrawings and discussed in the text.

[0165] While the present invention has been described herein withreference to particular embodiments thereof, a latitude of modification,various changes and substitutions are intended in the foregoingdisclosures, and it will be appreciated that in some instances somefeatures of the invention will be employed without a corresponding useof other features without departing from the scope and spirit of theinvention as set forth. Therefore, many modifications may be made toadapt a particular situation or material to the essential scope andspirit of the present invention. It is intended that the invention notbe limited to the particular embodiment disclosed as the best modecontemplated for carrying out this invention, but that the inventionwill include all embodiments and equivalents falling within the scope ofthe appended claims.

What is claimed is:
 1. A method of designing a racking configuration forboxes in racks and for determining which connections go betweendifferent racks, the method comprising: solving a rack selectoptimization sub-problem to determine racks to use; and solving a rackassign optimization sub-problem to determine which particular rack willhold a particular box.
 2. The method of claim 1, wherein theconfiguration is determined based upon an integer-programming solution.3. The method of claim 1, wherein the configuration is applicable to acomputer network system.
 4. The method of claim 1, where a connectionmay comprise one or more non-physical links formed from at least one of:a wireless link, a free-air/free-space optical link, a laser link, andan acoustic link.
 5. The method of claim 4, where a connection maycompriseone or more physical links formed from at least one of: anelectrical link, comprising at least one of a conductive wire, and acopper cabling; an optical fibre; and a mechanical link, comprising atleast one of a string, a rope, a tape, a wire, a pipe, a tube, achannel, and a hydraulic line.
 6. The method of claim 5, where at leastone of the connections is used as at least one of Ethernet link,InfiniBand link, Fibre Channel link, serial link, point-to-pointcomputer link, point-to-point network link, star computer link, starnetwork link, local area link, wide area link, campus-area link,metropolitan-area link, and a satellite link.
 7. The method of claim 1,further comprising: solving a rack layout optimization sub-problem todetermine a location of each rack that will hold at least one box. 8.The method of claim 1, further comprising: solving a box placementoptimization sub-problem to determine a placement of at least aparticular box in a particular rack.
 9. The method of claim 1, furthercomprising: selectively performing a local improvement heuristic inorder to improve the result from the actions of solving the optimizationsub-problems.
 10. The method of claim 5, wherein at least one of theactions of solving the optimization sub-problems is performed by use ofan optimization model.
 11. The method of claim 10, wherein theoptimization model to solve various sub-problems can be specified via aninteger programming language.
 12. The method of claim 5, wherein theaction of solving the rack select optimization sub-problem includes:solving the rack select optimization sub-problem based upon at least oneof the following constraints: a dimension of a box in a particular rackdoes not exceed a dimension of the rack, a selected rack meets the powerrequirements for boxes placed in the selected rack, a box requiringuninterrupted power (UPS) is placed in a rack with adequate UPScapability, heat generated by boxes in a single particular rack arewithin a particular limit, a selected rack meets another attributerequirement of a box that is placed in the selected rack, a certain typeof box and a certain type of rack are prohibited as a combination, aselected rack holds at least one box, a box and a backup of the box areplaced in separate racks, a box and a backup of the box are separated byat least some user-defined distance, and a box is assigned to acompatible rack.
 13. The method of claim 5 wherein the action of solvingthe rack select optimization sub-problem includes at least one of thefollowing objectives: to minimize the total cost of selected racks, tominimize the number of selected racks, to minimize the cost of racksthat are not yet owned, to minimize floor space needs, to minimize powerrequirements, to balance cooling requirements, to use a single racktype, and at least another type of objective.
 14. The method of claim 5,wherein the action of solving the rack assign optimization sub-problemincludes: solving the rack assign optimization sub-problem based upon atleast one of the following constraints: an associated physical attributeof each box, an associated wiring layout requirement of each box, arequirement that a particular box is to be placed in a particular rack,a requirement that a particular box is to not be placed in a particularrack.
 15. The method of claim 5, wherein the action of solving the rackassign optimization sub-problem includes: solving the rack assignoptimization sub-problem based upon at least one of the followingobjectives: to minimize the total link length, to minimize the totalnumber of inter-rack links, to minimize the number of links exceeding adefined length, to minimize the total length of inter-rack links, tominimize the cost of inter-rack links, to minimize the number of linkscrossing between machine rooms, domains, or buildings, to minimize theheight of the center of gravity of a rack, to leave space for futuregrowth, and slots for new boxes, to maximize the number of intra-racklinks, and to assign each box to a rack.
 16. The method of claim 7,wherein the action of solving the rack layout optimization sub-problemincludes: solving the rack layout optimization sub-problem based upon atleast one of the following objectives: to minimize the total linklength, to minimize the total number of inter-rack links, to minimizethe number of links exceeding a defined length, to minimize the cost ofinter-rack links, to minimize the total length of inter-rack links, tominimize the number of links crossing between machine rooms, domains, orbuildings, to minimize floor space needs, to minimize powerrequirements, to balance cooling requirements, and at least another typeof selectable objective.
 17. The method of claim 5 wherein the action ofsolving for the rack select optimization sub-problem includes: solvingfor a decision variable Y(r) to determine if a rack r is to be used tohold a box; and solving for a decision variable NX(bt,r) to determinethe number of boxes of type (bt) to be placed in a rack (r).
 18. Themethod of claim 5, wherein the action of solving for the rack assignoptimization sub-problem includes: solving for a decision variableX(a,r) to determine if a box (a) is to be assigned to a rack (r), andsolving for a decision variable X(b,s) to determine if a box (b) is tobe assigned to a rack (s); and solving for a decision variableZ(a,r,b,s) which is a quadratic term X(a,r)*X(b,s).
 19. The method ofclaim 18, wherein the variables of Z(a,r,b,s) are treated as continuousto increase the speed of the calculation process in the rack assignoptimization sub-problem.
 20. The method of claim 7, wherein the actionof solving for the rack layout optimization sub-problem includes:solving for a decision variable X(r,L) to determine if a rack (r) is tobe assigned to a location (L), and solving for a decision variableX(s,M) to determine if a rack (s) is to be assigned to a location (M);and solving for a decision variable Z(r,L,s,M) which is a quadratic termX(r,L)*X(s,M).
 21. The method of claim 20, wherein the variables ofZ(r,L,s,M) are treated as continuous to increase the speed of thecalculation process in the rack layout optimization sub-problem.
 22. Themethod of claim 5, wherein the racking configuration is subject to atleast one of the following constraints: a set of required boxes, a setof required links between the boxes, a set of available rack types, aset of locations for racks, at least one attribute for each box, atleast one attribute for each rack, at least one pre-assignment relatingto a rack and a box, and distances between locations of racks.
 23. Themethod of claim 22, wherein an attribute of a box includes at least oneof the following: device dimension, power requirement, uninterruptiblepower (UPS) requirement, cooling requirement, and at least anotherattribute.
 24. The method of claim 5, wherein the racking configurationis subject to at least one of the following objectives: to minimize thenumber of racks to be used, to minimize the total cost of racks to beused, to minimize the cost of remaining racks to be purchased, tominimize the cost, length or number of links between boxes in differentracks, and to minimize the number of long cables.
 25. The method ofclaim 5, wherein the actions of solving the optimization sub-problemsare performed sequentially.
 26. The method of claim 5, wherein theaction of solving the rack select optimization sub-problem includes:calculating bounds on the maximum number of each rack type to be used ina solution, and considering only those solutions adhering to the bounds.27. The method of claim 5, wherein the action of solving the rack selectoptimization sub-problem includes using a cost bound obtained from abin-packing method.
 28. The method of claim 5, wherein the action ofsolving the rack select optimization sub-problem includes: forcing atleast one requirement related to a box appearing in a given rack. 29.The method of claim 5, wherein the action of solving the rack selectoptimization sub-problem includes: requiring an anti-symmetry constraintso that a particular rack (r) will be required to be used for holding abox before a second rack (r+1) is used for holding a box.
 30. The methodof claim 5, wherein the action of solving the rack assign optimizationsub-problem includes: using an initial solution from the rack selectoptimization sub-problem for the rack assign optimization sub-problem.31. The method of claim 5, wherein the action of solving the rack assignoptimization sub-problem includes: using a lower bound from an initialsolution from the rack select optimization sub-problem when solving therack assign optimization sub-problem.
 32. The method of claim 5, whereinthe action of solving the rack assign optimization sub-problem includes:using constraint branching to increase the solution calculation processspeed in the rack assign optimization sub-problem.
 33. The method ofclaim 5, wherein the boxes form at least one network comprising: atleast one of a computer network, a universal data center, an Internetdata center, a telecommunications network, a storage network, a storagearea network (SAN), a local area network (LAN), a wide area network(WAN), a disk array, a tape library, a hub, a switch, a router, anothertype of network device, a UNIX system, and a WINDOWS system.
 34. Themethod of claim 5, wherein a box comprises at least one of: a computer,storage device, hub, switch, router, another type of network component,display, keyboard, another type of computer component, storage areanetwork (SAN) device, fan, air duct, telecommunication device, atelephone, another type of telephone equipment, a computer systemcomponent, Internet data center device, local area network (LAN) device,disk array component, tape library device, UNIX system component,WINDOWS system component, I/O subsystem, I/O controller, storagecontroller, power supply, cooling unit, and another type of electronicdevice.
 35. The method of claim 5, wherein the action of solving therack select optimization sub-problem comprises: defining ameta-constraint comprising a sum of particular constraints of boxesassigned to a particular rack; and assigning the boxes to the particularrack if the meta-constraint does not exceed a capacity for themeta-constraint of the particular rack.
 36. The method of claim 5,wherein the action of solving the rack assign optimization sub-problemcomprises: using the number of boxes assigned to each rack in the rackselect optimization sub-problem as a guideline in the rack assignoptimization sub-problem.
 37. The method of claim 37, wherein the actionof using the number of boxes comprises: in the rack select optimizationphase, choosing the number of instances of each rack type; in the rackselect optimization phase, choosing the number of boxes of each type toassign to each rack; and in the rack assign optimization phase, ensuringthat approximately the same number of boxes of each type are assigned toeach rack.
 38. A network system designed in accordance with the methodof claim
 1. 39. A method of selecting racks for holding boxes, themethod comprising: solving a rack select optimization sub-problem todetermine racks to use, where each rack has at least one rack attributeand each box has at least one device attribute, and where the rackattribute can accommodate the device attribute.
 40. A method ofassigning boxes to racks, the method comprising: solving a rack assignoptimization sub-problem to determine which particular racks will holdparticular boxes, where the particular racks are obtained from asolution to a rack select optimization sub-problem.
 41. A method ofassigning racks to locations, the method comprising: solving a racklayout optimization sub-problem to determine a location of each rack,where each rack is obtained from a solution to a rack selectoptimization sub-problem, and where particular racks are assigned tohold a particular box based upon a solution to a rack assignoptimization sub-problem.
 42. An apparatus for designing a rackingconfiguration and determining which connections go between differentracks of a system, the apparatus comprising: a machine-readablerepresentation for a racking configuration problem; and an solver thatcan read said machine-readable representation and that is configured to:solve a rack select optimization sub-problem to determine at least onerack to use; and solve a rack assign optimization sub-problem todetermine which particular rack will hold at least one box.
 43. Theapparatus of claim 42, wherein the solver can solve the sub-problems byuse an integer programming technique.
 44. The apparatus of claim 42,wherein the configuration is applicable to a computer network system.45. The apparatus of claim 42, where a connection may comprise one morenon-physical links formed from at least one of: a wireless link, afree-air/free-space optical link, a laser link, and an acoustic link.46. The apparatus of claim 42, where a connection may comprise one ormore physical links formed from at least one of: an electrical link,comprising at least one of a conductive wire, and a copper cabling; anoptical fibre; and an mechanical link, comprising at least one of astring, a rope, a tape, a wire, a pipe, a tube, a channel, and ahydraulic line.
 47. The apparatus of claim 42, where at least one of theconnections is used as at least one of Ethernet link, InfiniBand link,Fibre Channel link, serial link, point-to-point computer link,point-to-point network link, star computer link, star network link,local area link, wide area link, campus-area link, metropolitan-arealink, satellite link.
 48. The apparatus of claim 42, wherein the solveris configured to solve a rack layout optimization sub-problem todetermine a location of at least one rack that will hold at least onebox.
 49. The apparatus of claim 42, wherein the solver is configured tosolve a box placement optimization sub-problem to determine a placementof at least a particular box in a particular rack.
 50. The apparatus ofclaim 42, wherein a local improvement heuristic can be performed inorder to improve the result from the optimization sub-problems.
 51. Theapparatus of claim 42, wherein at least one of the actions of solvingthe optimization sub-problems is performed by use of an optimizationmodel.
 52. The apparatus of claim 51, wherein the optimization model tosolve various sub-problems can be specified via an integer programminglanguage.
 53. The apparatus of claim 46, wherein the rack selectoptimization sub-problem includes at least one of the followingconstraints: a dimension of a box in a particular rack does not exceed adimension of the rack, a selected rack meets the power requirements forboxes placed in the selected rack, a box requiring uninterrupted power(UPS) is placed in a rack with adequate UPS capability, heat generatedby boxes in a single particular rack are within a particular limit, aselected rack meets another attribute requirement of a box that isplaced in the selected rack, a certain type of box and a certain type ofrack are prohibited as a combination, a selected rack holds at least onebox, a box and a backup of the box are placed in separate racks, a boxand a backup of the box are separated by at least some user-defineddistance, a box is assigned to a compatible rack, and at least anothertype of constraint.
 54. The apparatus of claim 46 wherein the rackselect optimization sub-problem includes at least one of the followingobjectives: to minimize the total cost of selected racks, to minimizethe number of selected racks, to minimize the cost of racks that are notyet owned, to minimize floor space needs, to minimize powerrequirements, to balance cooling requirements, to use a single racktype, and at least another type of objective.
 55. The apparatus of claim46, wherein the solver can solve the rack assign optimizationsub-problem by: solving the rack assign optimization sub-problem basedupon at least one of the following constraints: an associated physicalattribute of each box, an associated wiring layout requirement of eachbox, a requirement that a particular box is to be placed in a particularrack, a requirement that a particular box is to not be placed in aparticular rack.
 56. The apparatus of claim 46, wherein the solver cansolve the rack assign optimization sub-problem by: solving the rackassign optimization sub-problem based upon at least one of the followingobjectives: to minimize the total link length, to minimize the totalnumber of inter-rack links, to minimize the number of links exceeding adefined length, to minimize the total length of inter-rack links, tominimize the cost of inter-rack links, to minimize the number of linkscrossing between machine rooms, domains, or buildings, to minimize theheight of the center of gravity of a rack, to leave space for futuregrowth, and slots for new boxes, to maximize the number of intra-racklinks, and to assign each box to a rack.
 57. The apparatus of claim 48,wherein the solver can solve the rack layout optimization sub-problemby: solving the rack layout optimization sub-problem based upon at leastone of the following objectives: to minimize the total link length, tominimize the total number of inter-rack links, to minimize the number oflinks exceeding a defined length, to minimize the cost of inter-racklinks, to minimize the total length of inter-rack links, to minimize thenumber of links crossing between machine rooms, domains, or buildings,to minimize floor space needs, to minimize power requirements, tobalance cooling requirements, and at least another type of selectableobjective.
 58. The apparatus of claim 46 wherein the solver can solvethe rack select optimization sub-problem by: solving for a decisionvariable Y(r) to determine if a rack r is to be used to hold a box; andsolving for a decision variable NX(bt,r) to determine the number ofboxes of type (bt) to be placed in a rack (r).
 59. The apparatus ofclaim 46, wherein the solver can solve the rack assign optimizationsub-problem by: solving for a decision variable X(a,r) to determine if abox (a) is to be assigned to a rack (r), and solving for a decisionvariable X(b,s) to determine if a box (b) is to be assigned to a rack(s); and solving for a decision variable Z(a,r,b,s) which is a quadraticterm X(a,r)*X(b,s).
 60. The apparatus of claim 59, wherein the variablesof Z(a,r,b,s) are treated as continuous to increase the speed of thecalculation process in the rack assign optimization sub-problem.
 61. Theapparatus of claim 46, wherein the solver can solve the rack layoutoptimization sub-problem by: solving for a decision variable X(r,L) todetermine if a rack (r) is to be assigned to a location (L), and solvingfor a decision variable X(s,M) to determine if a rack (s) is to beassigned to a location (M); and solving for a decision variableZ(r,L,s,M) which is a quadratic term X(r,L)*X(s,M).
 62. The apparatus ofclaim 61, wherein the variables of Z(r,L,s,M) are treated as continuousto increase the speed of the calculation process in the rack layoutoptimization sub-problem.
 63. The apparatus of claim 46, wherein theracking configuration is subject to at least one of the followingconstraints: a set of required boxes, a set of required links betweenthe boxes, a set of available rack types, a set of locations for racks,at least one attribute for each box, at least one attribute for eachrack, at least one pre-assignment relating to a rack and a box,distances between locations of racks, and at least another type ofconstraint.
 64. The apparatus of claim 46, wherein the rackingconfiguration is subject to at least one of the following constraints: aset of required boxes, a set of required links between the boxes, a setof available rack types, a set of locations for racks, at least oneattribute for each box, at least one attribute for each rack, at leastone pre-assignment relating to a rack and a box, and distances betweenlocations of racks.
 65. The apparatus of claim 46, wherein an attributeof a box includes at least one of the following: device dimension, powerrequirement, uninterruptible power (UPS) requirement, coolingrequirement, and at least another attribute.
 66. The apparatus of claim46, wherein the racking configuration is subject to at least one of thefollowing objectives: to minimize the number of racks to be used, tominimize the total cost of racks to be used, to minimize the cost ofremaining racks to be purchased, to minimize the cost, length or numberof links between boxes in different racks, to minimize the number oflong cables, and at least another type of objective.
 67. The apparatusof claim 46, wherein the solver solves the optimization sub-problemssequentially.
 68. The apparatus of claim 46, wherein the solver cansolve the rack select optimization sub-problem by: calculating bounds onthe maximum number of each rack type to be used in a solution, andconsidering only those solutions adhering to the bounds.
 69. Theapparatus of claim 46, wherein the solver can solve the rack selectoptimization sub-problem by using a cost bound obtained from abin-packing method.
 70. The apparatus of claim 46, wherein the solvercan solve the rack select optimization sub-problem by: forcing at leastone requirement related to a box appearing in a given rack.
 71. Theapparatus of claim 46, wherein the solver can solve the rack selectoptimization sub-problem by: requiring an anti-symmetry constraint sothat a particular rack (r) will be required to be used for holding a boxbefore a second rack (r+1) is used for holding a box.
 72. The apparatusof claim 46, wherein the solver can solve the rack assign optimizationsub-problem by: using an initial solution from the rack selectoptimization sub-problem for the rack assign optimization sub-problem.73. The apparatus of claim 46, wherein the solver can solve the rackassign optimization sub-problem by: using a lower bound from an initialsolution from the rack select optimization sub-problem when solving therack assign optimization sub-problem.
 74. The apparatus of claim 46,wherein the solver can solve the rack assign optimization sub-problemby: using constraint branching to increase the solution calculationprocess speed in the rack assign optimization sub-problem.
 75. Theapparatus of claim 46, wherein the boxes form at least one networkcomprising: at least one of a computer network, a universal data center,an Internet data center, a telecommunications network, a storagenetwork, a storage area network (SAN), a local area network (LAN), awide area network (WAN), a disk array, a tape library, a hub, a switch,a router, another type of network device, a UNIX system, and a WINDOWSsystem.
 76. The apparatus of claim 46, wherein a box comprises at leastone of: a computer, storage device, hub, switch, router, another type ofnetwork component, display, keyboard, another type of computercomponent, storage area network (SAN) device, fan, air duct,telecommunication device, a telephone, another type of telephoneequipment, a computer system component, Internet data center device,local area network (LAN) device, disk array component, tape librarydevice, UNIX system component, WINDOWS system component, I/O subsystem,I/O controller, storage controller, power supply, cooling unit, andanother type of electronic device.
 77. The apparatus of claim 46,wherein the solver can solve the rack select optimization sub-problemby: defining a meta-constraint comprising a sum of particularconstraints of boxes assigned to a particular rack; and assigning theboxes to the particular rack if the meta-constraint does not exceed acapacity for the meta-constraint of the particular rack.
 78. Theapparatus of claim 46, wherein the solver can solve the rack assignoptimization sub-problem by: using the number of boxes assigned to eachrack in the rack select optimization sub-problem as a guideline in therack assign optimization sub-problem.
 79. The apparatus of claim 78,wherein the action of using the number of boxes comprises: in the rackselect optimization phase, choosing the number of instances of each racktype; in the rack select optimization phase, choosing the number ofboxes of each type to assign to each rack; and in the rack assignoptimization phase, ensuring that approximately the same number of boxesof each type are assigned to each rack.
 80. An apparatus for designing aracking configuration for boxes in racks and for determining whichconnections go between different racks, the apparatus comprising: meansfor solving a rack select optimization sub-problem to determine racks touse; and coupled to the means for solving the rack select optimizationsub-problem, means for solving a rack assign optimization sub-problem todetermine which particular rack will hold a particular box.
 81. Theapparatus of claim 80, further comprising: coupled to the means forsolving a rack assign optimization sub-problem, means for solving a racklayout optimization sub-problem to determine a location of each rackthat will hold at least one box.
 82. The apparatus of claim 80, furthercomprising: coupled to the means for solving the rack layoutoptimization sub-problem, means for solving a box placement optimizationsub-problem to determine a placement of at least a particular box in aparticular rack.
 83. An article of manufacture, comprising: amachine-readable medium having stored thereon instructions to: solve arack select optimization sub-problem to determine racks to use; andsolve a rack assign optimization sub-problem to determine whichparticular rack will hold a particular box.
 84. The article ofmanufacture of claim 83, wherein the instructions can solve a racklayout optimization sub-problem to determine a location of each rackthat will hold at least one box.
 85. The article of manufacture of claim83, wherein the instructions can solve a box placement optimizationsub-problem to determine a placement of at least a particular box in aparticular rack.